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MEMORANDUM  FOR  ASSISTANT  SECRETARY  OF  THE  NAVY  (FINANCIAL 

MANAGEMENT  AND  COMPTROLLER) 

SUBJECT:  Audit  of  Functional  and  Physical  Configuration  Audits  of  the  Navy 
Tactical  Command  System-Afloat  (Project  No.  5AE-0032.04) 


Introduction 

We  are  providing  this  report  for  your  information  and  use.  This  report  is  the 
fifth  in  a  series  of  reports  resulting  from  our  audit  of  functional  and  physical 
configuration  audits  of  Defense  systems.  A  functional  configuration  audit  is  a 
formal  examination  of  functional  characteristics  of  test  data  for  configuration 
items  to  verify  that  the  items  have  achieved  their  specified  performance.  A 
physical  configuration  audit  is  a  formal  examination  to  verify  that  the 
configuration  items  "as  built"  conform  to  the  technical  documentation  that 
defines  the  item.  Enclosure  2  provides  definitions  of  technical  terms  used  in 
this  report. 


Audit  Results 

The  Navy  Tactical  Command  System-Afloat  (NTCS-A)  Program  Office 
adequately  managed  the  functional  and  physical  configuration  audit  process. 
Based  on  the  audit,  the  NTCS-A  Program  Office  began  updating  its 
configuration  management  plan  to  incorporate  current  functional  and  physical 
configuration  management  procedures.  NTCS-A  management  controls  were 
adequate. 


Audit  Objective 

The  audit  objective  was  to  evaluate  the  adequacy  of  the  functional  and  physical 
configuration  audit  processes  for  the  acquisition  of  the  NTCS-A.  Specifically, 
we  determined  whether  functional  and  physical  configuration  audits  verified  and 
documented  that  configuration  items  agreed  with  their  configuration 
identifications,  were  complete  and  accurate,  and  satisfied  program  requirements. 
We  also  evaluated  the  management  control  program  as  it  related  to  our  audit 
objective.  In  Enclosure  1,  we  discuss  the  scope  and  methodology  used  to 
accomplish  the  objective  as  well  as  management  controls  and  prior  audit 
coverage. 
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Audit  Background 

Navy  Tactical  Command  System-Afloat.  The  NTCS-A  is  an  acquisition 
category  II  program.  The  NTCS-A  provides  afloat,  Joint,  and  Allied 
Commanders  a  single,  integrated  command,  control,  and  intelligence  system. 
The  system  receives,  processes,  displays,  and  maintains  current  geo-locational 
information  on  friendly,  hostile,  and  neutral  land,  sea,  and  air  forces.  The 
NTCS-A  is  integrated  with  intelligence  and  environmental  information  in  all 
aspects  of  wartime  and  peacetime  missions  of  the  Navy.  In  total,  the  Navy 
plans  to  install  the  NTCS-A  on  251  ships  at  an  estimated  cost  of  $898  million. 
As  of  June  20,  1996,  the  Navy  had  installed  207  systems. 

Development  of  the  NTCS-A  began  in  1991,  when  the  Navy  integrated  seven 
existing  systems  to  form  the  NTCS-A.  Those  systems  were  the  Tactical  Flag 
Command  Center,  the  Afloat  Correlation  System,  the  Electronic  Warfare 
Coordination  Module  Subsystem,  the  Prototype  Ocean  Surveillance  Terminal, 
the  Joint  Operational  Tactical  System,  the  Fleet  Imagery  Support  Terminal,  and 
the  Naval  Intelligence  Processing  System.  Because  user  requirements  are 
dynamic,  the  NTCS-A  Program  Office  is  employing  an  evolutionary  acquisition 
strategy  to  develop  and  field  software  upgrades  of  the  NTCS-A  as  user 
requirements  are  refined. 

The  NTCS-A  Program  Office  acquires  nondevelopmental-item  hardware  and 
commercial  off-the-shelf  software  from  various  contractors  to  assemble  and 
upgrade  the  NTCS-A.  The  Research,  Development,  Test,  and  Evaluation 
Division  of  the  Naval  Command,  Control,  and  Ocean  Surveillance  Center,  San 
Diego,  California,  integrates  the  upgraded  software  into  the  NTCS-A. 
Enclosure  3  shows  a  diagram  of  the  NTCS-A. 

Configuration  Management  Guidance.  The  DoD  Regulation  5000.2-R, 
"Mandatory  Procedures  for  Major  Defense  Acquisition  Programs  (MDAPS)  and 
Major  Automated  Information  System  (MAIS)  Acquisition  Programs," 
March  15,  1996,  and  the  NTCS-A  Configuration  Management  Plan  contain 
configuration  management  guidance. 

DoD  Regulation.  The  DoD  Regulation  5000.2-R  establishes 
requirements  for  configuration  management.  Subpart  4.3,  "Systems 
Engineering,"  requires  that  the  program  manager  use  a  systems  engineering 
process  that  includes  configuration  management  to  control  the  system  products, 
processes,  and  related  documentation.  Further,  DoD  Regulation  5000.2-R 
mandates  that,  as  part  of  systems  engineering,  the  program  manager  should 
establish  a  configuration  management  process  to  identify,  document,  and  verify 
the  functional  and  physical  characteristics  of  an  item;  record  the  configuration 
of  an  item;  control  changes  to  an  item  and  its  documentation;  and  provide  a 
complete  audit  trail  of  decisions  and  design  modifications. 

Configuration  Management  Plan.  In  April  1992,  the  NTCS-A 
Program  Office  prepared  a  configuration  management  plan  (the  Plan).  The 
primary  objective  of  the  Plan  is  to  ensure  that  the  design,  development, 
production,  and  testing  of  the  NTCS-A  are  monitored,  controlled,  and 
substantiated  so  that  the  system  fulfills  its  specified  mission.  The  Plan 
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implements  the  Space  and  Naval  Warfare  Systems  Command  policy  for 
configuration  management  of  NTCS-A,  establishes  an  NTCS-A  Configuration 
Control  Board  and  subsystem  configuration  control  boards  (subsystem  boards), 
and  provides  policy  and  procedures  for  operation  of  those  control  boards  within 
the  NTCS-A.  The  Plan  discusses  technical  and  administrative  direction  and 
surveillance  to  identify  and  document  the  characteristics  of  designated 
configuration  items,  to  control  changes  to  those  characteristics,  and  to  record 
and  report  change  processing  and  implementation  status. 


Discussion 

The  NTCS-A  Program  Office  adequately  managed  the  functional  and  physical 
configuration  audit  process  and  tailored  its  configuration  management  process  to 
include  the  configuration  control  process  and  developmental  and  operational 
testing  results.  Based  on  our  audit,  the  NTCS-A  Program  Office  began 
updating  the  Plan  to  include  procedures  for  identifying,  documenting,  and 
verifying  the  functional  and  physical  characteristics  of  configuration  items  to 
conform  with  current  practices. 

Conducting  Functional  and  Physical  Configuration  Audits.  In  August  1991, 
the  NTCS-A  Program  Office  conducted  scheduled  functional  and  physical 
configuration  audits  of  the  NTCS-A  software  version  1.1.  The  configuration 
management  plan  required  that  configuration  audits  be: 

o  performed  in  accordance  with  Military  Standard  1521B,  "Technical 
Reviews  and  Audits  for  Systems,  Equipments,  and  Computer  Software," 
June  4,  1986,  which  established  guidance  for  conducting  functional  and  physical 
configuration  audits,  and 

o  scheduled  and  conducted  to  verify  that  the  configuration  of  each  item 
is  in  compliance  with  its  applicable  specifications,  drawings,  and  computer 
listings. 

NTCS-A  Program  Tailors  Requirement.  On  April  10,  1995,  the  Defense 
Standardization  Improvement  Council  canceled  Military  Standard  152  IB  and 
incorporated  the  functional  and  physical  configuration  audit  requirements  into 
Military  Standard  973,  "Configuration  Management,"  April  17,  1992.  Military 
Standard  973  requires  audits  of  configuration  items  to  verify  conformance  to 
specifications,  drawings,  interface  control  documents,  and  other  contract 
requirements. 

After  the  incorporation,  the  NTCS-A  Program  Office  decided  to  no  longer 
conduct  specific  functional  and  physical  configuration  audits.  Instead,  for 
subsequent  NTCS-A  software  versions,  the  NTCS-A  Program  Office  decided  to 
use  the  configuration  control  process  and  developmental  and  operational  testing 
results  to  identify,  document,  and  verify  the  functional  and  physical 
characteristics  of  configuration  items.  The  Program  Office,  however,  did  not 
update  the  NTCS-A  configuration  management  plan  to  show  its  tailored 
configuration  verification  process. 
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Configuration  Control  Process.  Configuration  control  involves  the  systematic 
coordination,  evaluation,  approval  or  disapproval,  and  release  of  approved 
changes  to  the  established  configuration  baseline.  The  NTCS-A  has  two  levels 
of  configuration  control:  the  NTCS-A  Configuration  Control  Board  and  the 
subsystem  boards. 

NTCS-A  Configuration  Control  Board.  The  NTCS-A  Configuration 
Control  Board  is  at  the  program  level  to  support  the  objectives  of  the  NTCS-A 
and  its  subsystems  configuration  change  control  program.  The  NTCS-A 
Configuration  Control  Board  evaluates  and  approves  change  requests  used  to 
document  deficiencies  or  recommended  improvements  and  changes  in  software, 
hardware,  documentation,  and  training.  If  approved,  the  change  request 
becomes  an  engineering  change  proposal. 

Subsystem  Boards.  The  subsystem  boards  are  located  at  the  Naval 
Command,  Control,  and  Ocean  Surveillance  Centers  in  San  Diego,  California, 
and  Portsmouth,  Virginia.  The  subsystem  boards  review  and  forward  change 
requests  to  the  NTCS-A  Configuration  Control  Board. 

Developmental  Testing.  The  Research,  Development,  Test,  and  Evaluation 
Division  of  the  Naval  Command,  Control,  and  Ocean  Surveillance  Center, 
San  Diego,  California,  is  the  independent  verification  and  validation  agent,  the 
developmental  test  and  evaluation  organization,  and  the  software  support 
organization  for  the  NTCS-A.  In  those  roles,  it: 

o  conducts  independent  reviews  of  the  software  product  for  functional 
effectiveness  and  technical  sufficiency; 

o  reviews,  evaluates,  and  monitors  the  contents  of  all  software 
specifications  and  software  tests; 

o  reviews  and  makes  recommendations  on  the  identification  of  computer 
software  configuration  items; 

o  conducts  acceptance  testing  of  software  releases; 

o  performs  maintenance  of  software  releases; 

o  provides  input  to  the  configuration  status  accounting  system  records; 

o  maintains  current  versions  of  software  descriptive  documentation  and 
software  media;  and 

o  participates  in  functional  and  physical  configuration  verifications. 

Operational  Testing.  The  Commander,  Operational  Test  and  Evaluation  Force 
(Test  Force),  Norfolk,  Virginia,  is  responsible  for  operational  test  and 
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evaluation  of  the  NTCS-A  and  for  making  recommendations  concerning 
whether  software  upgrades  of  the  NTCS-A  satisfy  user  operational 
requirements.  Specifically,  the  Test  Force: 

o  participates  in  integrated  product  teams  and  working  level  test 
planning  working  groups  to  develop  affordable  acquisition  and  test  strategies 
throughout  the  program  design  process  that  translate  operational  requirements 
into  a  system  solution; 

o  conducts  system  software  development  process  reviews  to  assess 
process  maturity  and  to  help  the  developing  agency  identify  areas  for  process 
improvement; 

o  plans  and  conducts  operational  tests; 

o  observes  developmental  testing  and  factory  acceptance  and 
qualification  testing  to  provide  early  feedback  to  the  developing  agency 
regarding  potential  shortcomings  in  system  effectiveness  of  suitability;  and 

o  reports  whether  a  system  is  operationally  effective,  suitable,  and 
survivable  before  fielding. 

The  Test  Force  conducted  its  latest  operational  evaluation  of  the  NTCS-A  from 
October  13  through  27,  1995.  The  purpose  of  the  evaluation  was  to  determine 
the  operational  effectiveness  and  operational  suitability  of  the  NTCS-A  software 
version  2.2.0.5  functioning  in  the  tactical  advanced  computer  (TAC)-3 
processor.  Additionally,  the  Test  Force  conducted  an  operational  assessment  of 
the  desktop  computer  processor  upgrade  and  the  TAC-4  processor  as  a 
workstation  using  the  version  2.2.0.5  software.  Based  on  its  operational 
evaluation,  the  Test  Force  concluded  that: 

o  the  NTCS-A  was  operationally  effective  and  suitable  as  upgraded  with 
the  version  2.2.0.5  software  functioning  in  the  TAC-3  processor,  and 

o  the  TAC-4  processor  and  the  desktop  computer  processor  upgrades 
were  potentially  operationally  effective  and  suitable. 

Identifying  and  Tracking  Corrective  Actions.  The  subsystem  boards  review, 
classify,  and  prioritize  NTCS-A  software,  documentation,  and  design  problems 
identified  during  developmental  and  operational  testing  as  well  as  fleet 
operations.  The  subsystem  boards  designate  the  deficiencies  as  action  items  and 
assign  them  to  the  Research,  Development,  Test,  and  Evaluation  Division  of  the 
Naval  Command,  Control,  and  Ocean  Surveillance  Center,  who  corrects  the 
action  items  immediately  or  during  future  software  upgrades.  The  Naval 
Command,  Control,  and  Ocean  Surveillance  Center  tracks  die  completion  of  the 
action  items  in  its  enhanced  automated  configuration  status  accounting  system. 

Software  Configuration  Status  Accounting  System.  In  late  1994,  the 
Research,  Development,  Test,  and  Evaluation  Division  of  the  Naval  Command, 
Control,  and  Ocean  Surveillance  Center  implemented  an  enhanced  automated 
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configuration  status  accounting  system  to  track  current  and  future  software 
deficiencies  for  the  NTCS-A.  That  automated  configuration  status  accounting 
system: 


o  records  and  reports  the  actual  status  of  change  requests  at  all  times, 

o  identifies  and  tracks  software  version  numbers  and  release  numbers 
comprising  a  software  build, 

o  notifies  the  change  requestor  when  a  correction  has  been  made,  and 

o  validates  software  releases  against  the  baseline. 

Before  implementation  of  the  automated  configuration  status  accounting  system, 
the  NTCS-A  Program  Office  could  not  demonstrate  that  the  Research, 
Development,  Test,  and  Evaluation  Division  of  the  Naval  Command,  Control, 
and  Ocean  Surveillance  Center  had  taken  appropriate  actions  to  correct  all 
NTCS-A  software  deficiencies  identified  before  1994  and  had  incorporated 
necessary  changes  into  the  NTCS-A  software  baseline. 

Tailored  Configuration  Verification  Process.  The  NTCS-A  Program  Office's 
revised  and  tailored  configuration  verification  process  conforms  with 
requirements  in  DoD  Regulation  5000.2-R.  Using  an  evolutionary  acquisition 
strategy  for  a  software  intensive  system,  the  NTCS-A  Program  Office 
demonstrated  that  functional  and  physical  characteristics  of  software  items  can 
be  effectively  verified  using  the  results  of  developmental  and  operational  tests 
and  operational  experience  in  the  fleet.  By  implementing  the  tailored 
configuration  verification  process,  the  NTCS-A  Program  Office  no  longer 
needed  to  have  specific  functional  and  physical  configuration  audits  of  software 
configuration  items. 


Conclusion 

Even  though  the  NTCS-A  Program  Office  no  longer  conducts  specific 
functional  and  physical  configuration  audits,  the  current  NTCS-A  configuration 
control  process  results  in  an  operationally  effective  and  suitable  NTCS-A 
configuration  that  satisfies  user  operational  requirements.  On  June  25,  1996, 
the  NTCS-A  Program  Office  advised  us  that  it  is  updating  its  configuration 
management  plan  to  incorporate  its  current  procedures  for  identifying, 
documenting,  and  verifying  the  functional  and  physical  characteristics  of 
configuration  items. 

We  request  that  the  NTCS-A  Program  Office  provide  us  with  a  copy  of  the 
updated  plan.  Because  of  the  success  of  the  NTCS-A  Program  Office  in 
effectively  tailoring  the  configuration  control  process  for  a  software  intensive 
system  that  uses  an  evolutionary  acquisition  strategy,  the  NTCS-A  Program 
Manager  should  submit  the  plan  to  the  Deputy  Under  Secretary  of  Defense 
(Acquisition  Reform)  to  be  included  in  the  Defense  Acquisition  Deskbook  as  an 
example  of  a  way  to  tailor  the  configuration  management  process  for  a  software 
intensive  system  that  uses  an  evolutionary  acquisition  strategy. 
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Management  Comments 

We  provided  a  draft  of  this  report  to  you  on  July  29,  1996.  Because  the  report 
contains  no  findings  or  recommendations,  written  comments  were  not  required, 
and  none  were  received.  Therefore,  we  are  publishing  this  report  in  final  form. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  If  you  have  questions 
on  this  report,  please  contact  Mr.  John  E.  Meling,  Audit  Program  Director,  at 
(703)  604-9091  (DSN  664-9091)  or  Mr.  Jack  D.  Snider,  Audit  Project 
Manager,  at  (703)  604-9087  (DSN  664-9087).  Enclosure  5  lists  the  distribution 
of  this  report.  The  audit  team  members  are  listed  inside  the  back  cover. 

David  K.  Steensma 
Deputy  Assistant  Inspector  General 
for  Auditing 

Enclosures 
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Scope  and  Methodology 


This  appendix  discusses  the  scope  and  methodology  used  to  accomplish  the 
objective  as  well  as  management  controls  and  prior  audit  coverage. 


Scope 


We  conducted  this  audit  from  January  through  June  1996  and  reviewed  data 
dated  from  August  1988  through  June  1996.  To  accomplish  the  objective,  we: 

o  reviewed  the  minutes  of  the  functional  and  physical  configuration 
audits  conducted  on  the  NTCS-A  and  the  action  items  generated  during  those 
audits; 


o  reviewed  contracts  and  other  contracting  documents,  including 
determination  and  findings,  acquisition  plans,  technical  direction  letters,  and 
delivery  orders; 

o  reviewed  configuration  status  accounting  system  documentation  and 
reports; 

o  reviewed  configuration  management  plans,  test  plans,  and 
developmental  and  operational  test  reports;  and 

o  discussed  issues  relating  to  the  effectiveness  of  the  functional  and 
physical  configuration  audit  process  with  personnel  from  the  Office  of  the 
Secretary  of  Defense  and  with  program,  technical,  and  contracting  officials  at 
the  NTCS-A  Program  Office;  the  Space  and  Naval  Warfare  Systems  Command; 
the  Research,  Development,  Test,  and  Evaluation  Division  and  the  In-Service 
Engineering  Division,  East  Coast,  of  the  Naval  Command,  Control,  and  Ocean 
Surveillance  Center;  and  the  Commander,  Operational  Test  and  Evaluation 
Force.  Enclosure  4  lists  the  organizations  visited  or  contacted. 


Methodology 

We  conducted  this  program  audit  in  accordance  with  auditing  standards  issued 
by  the  Comptroller  General  of  the  United  States,  as  implemented  by  the 
Inspector  General,  DoD.  We  included  such  tests  of  management  controls  as  we 
deemed  necessary.  We  did  not  rely  on  computer-processed  data  to  develop 
conclusions  on  this  audit.  Technical  experts  from  the  Technical  Assessment 
Division  of  the  Analysis,  Planning,  and  Technical  Support  Directorate, 
Inspector  General,  DoD,  and  a  consultant  from  the  Acquisition  Management 
Directorate,  Inspector  General,  DoD,  assisted  in  the  review  of  functional  and 
physical  configuration  audit  software  documentation  and  contracting  procedures, 
respectively. 


Enclosure  1 
(Page  1  of  2) 


Scope  and  Methodology 


Management  Control  Program 

DoD  Directive  5010.38,  "Internal  Management  Control  Program,"  April  14, 
1987,  requires  DoD  organizations  to  implement  a  comprehensive  system  of 
management  controls  that  provides  reasonable  assurance  that  programs  are 
operating  as  intended  and  to  evaluate  the  adequacy  of  the  management  controls. 

Scope  of  Review  of  Management  Control  Program.  We  limited  our  review 
because  of  relevant  coverage  in  Inspector  General,  DoD,  Report  No.  96-028, 
"Implementation  of  the  DoD  Management  Control  Program  for  Major  Defense 
Acquisition  Programs,"  November  28,  1995.  The  report  discussed  the 
effectiveness  of  die  management  control  program  that  the  Defense  Acquisition 
Executive  and  the  Component  Acquisition  Executives  used  for  major  Defense 
acquisition  programs.  'Die  report  concluded  that  the  acquisition  community  had 
not  effectively  integrated  DoD  Management  Control  Program  requirements  into 
its  management  assessment  and  reporting  processes.  As  a  result  of  the  report 
recommendations,  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology  integrated  DoD  Directive  5010.38  requirements  into  the  March  15, 
1996,  revision  to  DoD  Directive  5000.1,  "Defense  Acquisition,"  and  DoD 
Regulation  5000.2-R.  Acquisition  managers  are  now  to  use  program  cost, 
schedule,  and  performance  parameters  as  control  objectives  to  implement  the 
DoD  Directive  5010.38  requirements.  The  managers  are  to  identify  material 
weaknesses  through  deviations  from  approved  acquisition  program  baselines  and 
exit  criteria  in  the  Defense  Acquisition  Executive  Summary  report. 
Accordingly,  we  limited  our  review  to  management  controls  over  the  functional 
and  physical  configuration  audit  process  at  the  NTCS-A  Program  Office.  We 
did  not  assess  management's  self-evaluation  of  those  controls. 

Adequacy  of  Management  Controls.  In  1992,  the  NTCS-A  Program  Office 
conducted  a  management  control  review.  The  Program  Office  determined  that 
management  controls  were  in  place  and  the  risk  was  low.  However,  the 
management  control  review  did  not  specifically  cover  functional  and  physical 
configuration  audits  as  part  of  an  assessable  unit.  Even  though  the  Program 
Office  did  not  evaluate  functional  and  physical  configuration  audits,  we  did  not 
identify  a  material  management  control  weakness  for  configuration  audits. 


Prior  Audit  Coverage 

During  the  last  5  years,  the  General  Accounting  Office;  the  Office  of  the 
Inspector  General,  DoD;  and  the  Naval  Audit  Service  have  not  issued  reports  on 
the  NTCS-A  addressing  functional  and  physical  configuration  audit  issues. 
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Acquisition  Categories.  Categories  established  to  provide  decentralized 
decisionmaking  and  execution  and  compliance  with  statutorily  imposed 
requirements.  The  categories  consist  of  I,  major  Defense  acquisition  programs; 
IA,  major  automated  information  systems;  II,  major  systems;  and  III,  all  other 
acquisition  programs. 

Configuration  Control  Board.  A  Government  or  contractor  board  composed 
of  technical  and  administrative  representatives  who  recommend  approval  or 
disapproval  of  proposed  engineering  changes  to  a  configuration  item's  current 
approved  configuration  documentation.  The  board  also  recommends  approval 
or  disapproval  of  proposed  waivers  and  deviations  from  a  configuration  item's 
current  approved  configuration  documentation. 

Configuration  Identification.  The  process  of  establishing  and  describing  the 
contractual  baselines  and  related  configuration  items. 

Configuration  Item.  An  aggregation  of  hardware,  firmware,  or  computer 
software  or  any  of  their  discrete  portions  that  satisfies  an  end  use  function  and  is 
designated  by  the  Government  for  separate  configuration  management. 

Configuration  Management.  Technical  and  administrative  direction  and 
surveillance  actions  taken  to  identify  and  document  functional  and  physical 
characteristics  of  an  item,  to  control  changes  to  a  item  and  its  characteristics, 
and  to  record  and  report  change  processing  and  implementation  status. 

Configuration  Management  Plan;  A  document  defining  how  configuration 
management  will  be  implemented,  including  policies  and  procedures,  for  a 
particular  acquisition  or  program. 

Configuration  Status  Accounting  System.  A  system  implemented  to  record 
and  report  information  needed  to  manage  configuration  items  effectively, 
including  a  record  of  the  approved  configuration  documentation  and 
identification,  the  status  of  proposed  changes  to  that  configuration,  and  the 
implementation  status  of  approved  changes.  The  purpose  of  a  configuration 
status  accounting  system  is  to  effectively  manage  the  configuration  of  a  system 
and  ensure  accurate  identification  of  each  configuration  item  and  delivered  unit. 

Engineering  Change  Proposal.  A  contractor  document  describing  and 
justifying  a  proposed  engineering  change  and  applicable  costs  that  is  submitted 
to  the  Government  for  approval  or  disapproval. 

Evolutionary  Acquisition.  An  acquisition  strategy  used  to  procure  a  system 
that  evolves  during  development  to  achieve  an  overall  system  capability.  An 
underlying  factor  in  evolutionary  acquisition  is  the  need  to  field  a  well-defined 
core  capability  quickly  in  response  to  a  validated  requirement,  while  planning 
through  an  incremental  upgrade  program  to  eventually  enhance  the  system  to 
provide  the  overall  system  capability. 
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Firmware.  The  combination  of  a  hardware  device  and  computer  instructions  or 
computer  data  that  reside  as  read-only  software  on  the  hardware  device. 

Functional  Configuration  Audit.  A  formal  examination  of  functional 
characteristics  of  test  data  for  configuration  items  to  verify  that  the  item  has 
achieved  the  performance  specified  in  its  functional  or  allocated  identification. 
If  the  item  was  developed  at  Government  expense,  the  functional  configuration 
audit  must  be  performed  before  acceptance  of  the  item.  The  functional 
configuration  audit  must  be  performed  on  a  prototype  or  the  configuration  to  be 
released  for  production  of  the  operational  quantities. 

Integrated  Product  and  Process  Development.  A  management  process  that 
integrates  all  activities  from  product  concept  through  production  and  field 
support,  using  multidisciplinary  teams  to  simultaneously  optimize  the  product 
and  its  manufacturing  and  supportability  to  meet  cost  and  performance 
objectives. 

Low-Rate  Initial  Production.  The  production  of  a  system  in  limited  quantity 
to  provide  articles  for  operational  test  and  evaluation  and  to  establish  an  initial 
production  rate  sufficient  to  lead  to  full-rate  production  upon  successful 
completion  of  operational  testing. 

Nondevelopmental  item.  Any  item  of  supply  that: 

o  is  available  in  the  commercial  marketplace; 

o  is  in  use  by  an  agency  or  department  of  the  United  States  with  which 
the  United  States  has  a  mutual  Defense  cooperation  agreement; 

o  requires  only  minor  modification  to  meet  the  requirements  of  the 
procuring  agency;  or 

o  is  currently  being  produced,  but  is  not  in  use  or  available  in  the 
commercial  marketplace. 

Physical  Configuration  Audit.  A  formal  examination  to  verify  that  the 
configuration  item  "as  built"  conforms  to  the  technical  documentation  that 
defines  the  item.  The  physical  configuration  audit  includes  a  detailed  audit  of 
engineering  drawings,  specifications,  technical  data,  and  tests  used  in 
production  of  the  item.  The  physical  configuration  audit  may  be  conducted  on 
the  first  full-rate  production  or  the  first  low-rate  initial  production  item. 
Government  program  office  approval  of  the  product  specification  and 
satisfactory  completion  of  the  physical  configuration  audit  establishes  the 
product  baseline.  A  contractor  is  required  to  process  all  subsequent  changes  to 
the  product  baseline  by  the  formal  engineering  change  proposal  process. 


II 


Enclosure  2 
(Page  2  of  3) 


Definitions  of  Technical  Terms 


Product  Baseline.  The  initially  approved  documentation  describing: 

o  all  necessary  functional  and  physical  characteristics  of  the 
configuration  item; 

o  any  required  joint  and  combined  operations; 

o  the  selected  functional  and  physical  characteristics  designed  for 
production  acceptance  testing;  and 

o  tests  necessary  for  deployment,  installation,  support,  training,  and 
disposal  of  the  configuration  item. 

Prototype.  An  original  or  model  on  which  a  later  item  is  formed  or  based. 

Specifications.  A  document  intended  primarily  for  use  in  procurement  that 
clearly  and  accurately  describes  the  essential  technical  requirements  for  items, 
materials,  or  services,  including  the  procedures  for  determining  whether  the 
requirements  have  been  met. 
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Single  Monitor 
Workstation 


Remote  Processor 
Rack 


Organizations  Visited  or  Contacted 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology,  Washington,  DC 
Principal  Deputy  Under  Secretary  of  Defense  (Acquisition  and  Technology), 
Washington,  DC 

Computer-Aided  Acquisition  and  Logistic  Support,  Alexandria,  VA 
Defense  Standardization  Improvement  Council,  Alexandria,  VA 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition), 
Washington,  DC 

Naval  Information  Systems  Management  Center,  Arlington,  VA 
Space  and  Naval  Warfare  Systems  Command,  Arlington,  VA 

Command,  Control,  Communications,  Computers,  and  Intelligence  Directorate, 
Arlington,  VA 

Navy  Tactical  Command  System-Afloat  Program  Office,  Arlington,  VA 
Naval  Command,  Control,  and  Ocean  Surveillance  Center,  San  Diego,  CA 
Research,  Development,  Test,  and  Evaluation  Division,  San  Diego,  CA 
In-Service  Engineering  Division,  East  Coast,  Portsmouth,  VA 
Commander,  Operational  Test  and  Evaluation  Force,  Norfolk,  VA 


Other  Defense  Organization 

Defense  Logistics  Agency,  Fort  Belvoir,  VA 
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Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 
Deputy  Under  Secretary  of  Defense  (Acquisition  Reform) 
Director,  Defense  Logistics  Studies  Information  Exchange 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Assistant  to  the  Secretary  of  Defense  (Public  Affairs) 


Department  of  the  Army 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 

Assistant  Secretary  of  the  Navy  (Research,  Development,  and  Acquisition) 
Commander,  Space  and  Naval  Warfare  Systems  Command 

Program  Director,  Command,  Control,  Communications,  Computers  and 
Intelligence 

Program  Manager,  Navy  Tactical  Command  System  -  Afloat  Program  Office 
Auditor  General,  Department  of  the  Navy 

Commander,  Naval  Command,  Control,  and  Ocean  Surveillance  Center 
Program  Manager,  Research,  Development,  Test,  and  Evaluation  Division 
Program  Manager,  In-Service  Engineering  Division,  East  Coast 
Commander,  Operational  Test  and  Evaluation  Force 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 


Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Logistics  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 
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Non-Defense  Federal  Organizations  and  Individuals 

Office  of  Management  and  Budget 

Technical  Information  Center,  National  Security  and  International  Affairs  Division, 
General  Accounting  Office 

Chairman  and  ranking  minority  member  of  each  of  the  following  congressional 
committees  and  subcommittees: 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  National  Security,  Committee  on  Appropriations 
House  Committee  on  Government  Reform  and  Oversight 
House  Subcommittee  on  National  Security,  International  Affairs,  and  Criminal 
Justice,  Committee  on  Government  Reform  and  Oversight 
House  Committee  on  National  Security 
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The  Acquisition  Management  Directorate,  Office  of  the  Assistant  Inspector 
General  for  Auditing,  DoD,  prepared  this  report. 

Patricia  A.  Brannin 
John  E.  Meling 
Jack  D.  Snider 
Cordelia  Grace-Scott 
Debra  A.  Calhoun 
Lisa  M.  Waller 
Gregory  R.  Donnellon 
Garry  D.  Durfey 
Samuel  H.  Metzler 
Noelle  G.  Blank 
Teresa  D.  Bone 
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